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SUMMARY 


Above Real-Time Training (ARTT) is the training acquired on a real time simulator when it 
is modified to present events at a faster pace than normal. The experiments related to training of 
pilots performed by NASA engineers (Kolf in 1973, Hoey in 1976) and others (Guckenberger, 
Crane and their associates in the nineties) have shown that in comparison with the real time 
training (RTT), ARTT provides the following benefits: Increased rate of skill acquisition, 
reduced simulator and aircraft training time, and more effective training for emergency 
procedures. 

Two sets of experiments have been performed; they are reported in professional conferences 
and the respective papers are included in this report. The retention of effects of ARTT has been 
studied in the first set of experiments and the use of ARTT as top-off training has been examined 
in the second set of experiments. In ARTT, the pace of events was 1 .5 times the pace in RTT. In 
both sets of experiments, university students were trained to perform an aerial gunnery task. The 
training unit was equipped with a joystick and a throttle. The student acted as a nose gunner in a 
hypothetical two place attack aircraft. The flight simulation software was installed on a Universal 
Distributed Interactive Simulator platform supplied by ECC International of Orlando, Florida. 

In the first set of experiments, two training programs RTT or ARTT were used. Students 
were then tested in real time on more demanding scenarios: either immediately after training or 
two days later. The effects of ARTT did not decrease over a two day retention interval and 
ARTT was more time efficient than real time training. Therefore, equal test performance could 
be achieved with less clock-time spent in the simulator. 

In the second set of experiments three training programs RTT or ARTT or RARTT, were 
used. In RTT, students received 36 minutes of real time training. In ARTT, students received 36 
minutes of above real time training. In RARTT, students received 18 minutes of real time 
training and 18 minutes of above real time training as top-off training. Students were then tested 
in real time on more demanding scenarios. The use of ARTT as top-off training after RTT 
offered better training than RTT alone or ARTT alone. It is, however, suggested that a similar 
experiment be conducted on a relatively more complex task with a larger sample of participants. 

Within the proposed duration of the research effort, the setting up of experiments and trial 
runs on using ARTT for team training were also scheduled but they could not be accomplished 
due to extra ordinary challenges faced in developing the required software configuration. Team 
training is, however, scheduled in a future study sponsored by NASA at Tuskegee University. 
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1.1 ABSTRACT 

Above real-time training (ARTT) is an instructional strategy in which the pace of events in a 
real-time simulation is increased. University students were trained to perform an aerial gunnery 
task either in real-time or at 1.5 times real-time (ARTT). Subjects were then tested in real-time 
on more demanding scenarios either immediately after training, or two days later. Results show 
that students trained in real-time or using ARTT performed equally well on test trials and that the 
effects of ARTT did not decay more rapidly than real-time training. Further, students trained 
using ARTT required less clock time to achieve equal test performance as students trained using 
real-time simulation. 

1.2 RETENTION OF EFFECTS OF ABOVE REAL TIME TRAINING 

Above real-time training (ARTT) is an instructional tool for use in real-time simulators. Kolf 
(1973) and Hoey (1976) document applications of ARTT at the National Aeronautics and Space 
Administration's (NASA) Dryden Flight Research Center that were aimed at improving test 
pilots' ability to keep up with the pace of events in test flights. Kolf notes that, "regardless of the 
type or amount of pre-flight simulator training accomplished by the pilot, the actual flight seems 
to take place in a much faster time frame than real time," (p. 1). Hoey (1976) reports that in the 
X-15 program, pilots typically spent ten hours in the simulator for each ten minutes of flight. 
Even with this preparation, pilots reported that, "It sure seems to happen faster in the real 
airplane," or, "I had the feeling that I was 'behind the airplane' ", (pp. 2 - 3). Hoey (1976) 
describes application of ARTT to a flight test program for remotely piloted vehicles (RPV). In 
this case, the training environment was exactly the same as the actual flight environment. 
Nevertheless, RPV pilots who used simulation at 1 .4 times real time as final preparation before a 
flight reported being, "Less rushed and more confident," (p. 18) than when using real-time 
training exclusively. A typical practice was to conduct 70% of training at real time with the last 
30% at 1 .4 times real time. 

More recently, Schneider, Vidulich, & Yeh (1982) and Vidulich, Yeh, & Schneider (1983) used 
time-compression in developing training systems for air traffic controllers. The task for these 
controllers was to monitor an aircraft's flight path on a radar display. Actual aircraft would 
traverse 20 nautical miles and require approximately five minutes to complete an assigned turn. 
These researchers increased the apparent rate of time in the simulator to 20 times real time so 
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that a turn would be completed in approximately 15 seconds. Vidulich, Yeh, & Schneider (1983) 
trained university students over four hours to perform a turn-point initiation task. A group of 
students who performed the task in real time experienced approximately 32 trials in four hours of 
training. A group performing the same task using ARTT at 20 times real time received 
approximately 260 time-compressed trials followed by only three or four real-time trials in four 
hours of training. All trainees were tested at real time for two hours. ARTT subjects showed 
significantly better performance at initiating turns properly. These authors assert that the ARTT 
improves training effectiveness by allowing many trials and training under a mild speed stress. 

Guckenberger, Uliano, Lane, & Stanney (1993) conducted an experiment using 24 experienced 
F-16C pilots trained at real time or ARTT. All pilots then tested at real time. ARTT groups 
showed faster threat response than the group trained in real time and achieved significantly more 
bandit kills during real-time, test trials. Schneider (1989) proposed that the primary effect of time 
compression is to allow more training trials within a given period of clock time. In the air traffic 
control studies, subjects were given the same amount of training time in the simulator so that the 
ARTT subjects received more training trials. In contrast, Guckenberger et al. gave all subjects 
the same number of training trials so that the ARTT subjects received less training time than the 
students trained in real time. Since the pilots trained using ARTT performed better on real-time 
test trials than students trained in real time, Guckenberger et al.'s results indicate that ARTT has 
a beneficial effect beyond simply increasing the number of training events. 

Crane and Guckenberger (1997) conducted two experiments to evaluate applications of ARTT 
for training air combat skills and emergency procedures. In the first experiment, experienced Air 
Force F-16 pilots practiced emergency procedures and air intercepts using conventional, real- 
time simulation or ARTT at 1.5 times real time. The pilots trained using ARTT received the 
same number of training trials but less clock time in the simulator as pilots trained in real time. 
All pilots were then tested in real time. Pilots trained using ARTT performed emergency 
procedures and defeated bandit aircraft significantly faster than pilots trained in real time. In the 
second experiment, student F-16 pilots practiced using air-to-air radar in real time or ARTT. 
Students trained using ARTT received more training trials in approximately the same amount of 
clock time as the students trained in real time. ARTT students performed better on a more 
complex real-time test than students trained in real-time. Crane and Guckenberger concluded that 
ARTT at 1.5 times real-time is more time efficient than conventional, real-time simulation and 
can improve performance by allowing more training events to be experienced within a given 
period of simulator time. ARTT also supported better real-time test performance for tasks 
requiring skilled performance in time and workload management. 

In previous ARTT experiments, real-time transfer tests were conducted immediately after 
training. One hypothesis to explain the effect of ARTT is that trainees have adapted to a rapid 
pace of responding and, like sensory adaptation, this effect will rapidly decay with time. To 
assess whether effects of ARTT will be retained over a period typical of many training 
environments, performance of students using ARTT or real-time training was tested immediately 
after training or two days later. In addition, previous studies have been conducted using either a 
fixed number of training trials with ARTT trainees receiving less clock time in the simulator or, 
a fixed amount of clock time in the simulator with ARTT trainees receiving more training trials. 
Subjects in the present experiment were trained varying both simulated time and the number of 
training trials. 
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1.3 METHOD 

This experiment used a 2 X 2 X 2 between subjects factorial design. The independent variables 
were real-time training (RT) vs. ARTT, 10 training trials vs. 15 training trials, and O-day 
retention vs. 2-day retention. Participants were randomly assigned to conditions in blocks of 
eight (i.e., one per condition). The experiment was conducted in three phases, familiarization, 
training and testing. For all participants, regardless of condition, familiarization and testing were 
conducted in real time. Only training trials involved real time vs. above real time. 

Participants 

Thirty-two undergraduate, university students (25 males, and 7 females) volunteered to 
participate. Volunteers received no financial remuneration or extra credit for their participation. 

Equipment 

The equipment used by the students consisted of four personal computers on Local Area 
Network (LAN). The software system produced by ECC International of Orlando, Florida is 
called Universal Distributed Interactive Simulation (UDIS), which is a turnkey DIS environment. 
The computers were configured as two student stations and two instructor-operator stations. 

Student Stations. The student stations were equipped with a video monitor and joystick on the 
right and a throttle on the left. For this experiment, the student stations were configured as if the 
student were a nose gunner in a hypothetical two-place attack aircraft. The UDIS student stations 
operate in RT or ARTT as selected from the instructor's station. The virtual world in which the 
students were operating was created from digital terrain elevation data. Textures were then 
overlaid on top of the terrain to create out-the-window visual imagery. 

Instructor-Operator Stations. Each IOS was used to initialize student stations and to provide real- 
time monitoring of student actions and interactions. The IOS also incorporated an interactive 
playback utility (IPU), which was used to play back previously recorded aircraft flight paths. 
Ownship flight paths plus the actions of friendly and enemy aircraft in this experiment were 
recorded and replayed using the IPU for the student acting as the gunner. 

Flight Scenarios. Separate, three-minute flight scenarios were recorded for familiarization, 
training, and test. Scenarios used for familiarization were designed to be relatively easy and met 
the following criteria: 

There were approximately 15 targets presented during the flight. 

Each target were exposed for approximately eight seconds 
Target aircraft were in level flight at right angles and co- altitude to ownship 
No more than one target was in view at any time 
• The student's attack aircraft flew straight and level 

Three emergency counter-measures (ECMs) were required. 

Training flights consisted of two types: moderate or difficult; characteristics of these flights are 
described in Table 1. During training, all participants received three of the difficult flights, 
randomly ordered and placed within their prescribed number of trials. The remaining flights were 
of moderate difficulty. Testing consisted of three different difficult flights. 
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Table 1.1 Characteristics of moderate and difficult scenarios 


Scenario variables 

Type of scenario 


Moderate 

Difficult 

Number of targets presented 
within three minute flight 

19-24 

39-43 

Duration of each target's 
exposure 

5*7 seconds 

4-5 seconds 

Target aspect 

Level flight at right angles and 
co-altitude to ownship 

Targets at all headings and 
altitudes 

Number of targets in view at 
any moment 

<2 

<4 

Ownship motion 

Turns < 90° of heading change 
with<45 bank 

Turns up to 1 80 with 90 bank 
plus barrel rolls 

ECM responses required 

4-6 

6-9 


Procedure 

After signing an informed consent agreement, participants were asked to complete a one- 
page background survey, which asked questions regarding handedness, age, gender, and number 
of hours of video games played per week, as well as experience in marksmanship. 

Familiarization. Participants were seated in front of the computer monitor and informed that 
they would receive familiarization instructions that would describe the task and the use of the 
computer and its controls. The head-up display (HUD) that would be present during the actual 
flights was demonstrated. The HUD included symbology indicating whether a gun or missile was 
activated, and the slewable weapon's cursor. Their mission was to, "act as a nose gunner on an 
attack aircraft whose mission is to fire a missile at a large transport aircraft that is flying near an 
airport." The arrack aircraft's flight path was previously recorded and not under student control. 
The student's joystick slewed the weapon's cursor; the throttle control was not functional 
although buttons on the throttle were used in the ECM task. Students were also instructed that en 
route to the airport they would encounter a number of threats. Their task was to identify the type 
of threat, and determine whether a missile or gun would be required to destroy it. Helicopters 
required a gun, deployed by a trigger squeeze, whereas airplanes, including the transport aircraft, 
required a missile deployed by a button press. A flip of a switch on the joystick would change 
the weapon from guns to missiles. In addition, participants were required to determine whether 
the aircraft was a bandit or a friendly aircraft by its color; friendly targets were dark colored 
helicopters and light colored airplanes while bandits were light colored helicopters and dark 
colored planes. They were instructed to shoot all enemy aircraft and not to shoot friendlies. 

Participants were also informed of two other controls, the "identify friend or foe" (IFF) 
button, and the ECM buttons. Specifically, if they needed help in identifying an aircraft, they 
could push the IFF button and after a two-second delay would hear an audio signal identifying 
the aircraft. Friendly targets produced a harsh sound, while bandit targets produced a pleasant 
ding ding sound. Participants also were told that when a bandit has locked on to them, they 
would hear a high-pitched beeping sound. They were to employ ECM immediately by pushing 
one pinky button under the trigger on the right handle and then pushing the large round gray 
button on the left handle. For all procedures, the experimenter requested that the student perform 
the procedure after describing it. 
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Participants were informed of point values for each procedure or task; points could be gained 
by shooting enemies with the correct weapon: +2, proper ECM activations: +1, and destroying 
the C-17: +5. Points could be lost by killing a friendly: -4, missing a bandit: -2, and killing a 
bandit with the wrong weapon: -2. Scores were summed, divided by total possible, points for that 
flight, and multiplied by 100. If all procedures were carried out correctly, the maximum score 
would be 100%. Participants were told that when each flight ended, ownship position would 
freeze and feedback regarding performance would be displayed on the screen. At this point, the 
instructor would review the performance and then start the next trial. 

Participants were then instructed to put on their headsets and prepare for the first 
familiarization trial by placing their hands on the proper controls. The instructor stated that on 
the first trial, he or she would coach the participant through the first half of the flight. After that 
point they would be available to answer questions. Coaching consisted of identifying the target 
for the participant, and then instructing on which weapon to use. When an ECM event was about 
to occur, the instructor told the participant to be prepared for it, and if the participant had trouble 
in executing the procedure, the instructor described it and/or pointed to the controls for him or 
her. All participants received the same flight for their first familiarization flight. During the 
remainder of familiarization, the instructor simply answered questions at the end of each flight 
and reviewed the participant's performance, identifying what the participant had done correctly 
and incorrectly. Participants continued receiving familiarization trials selected randomly until 
they had reached at least 30% correct on two consecutive trials. 

Training . Training proceeded in the same fashion as familiarization, except that some 
subjects received ARTT for all training trials, while others received RT training. In addition, 
some subjects received ten trials while others received fifteen. Also, there was no verbal contact 
between participant and instructor during a trial, but feedback was reviewed after each trial. 

Testing . Prior to testing, participants were told that on the next few flights they would not 
receive feedback regarding their performance until all three flights were completed. Participants 
in the zero delay condition were asked to take a five-minute break between training and testing. 
Participants in the two-day retention condition were told that they were finished for the day. 
They were reminded of their time to return, and asked to please return as scheduled. At the 
completion of testing, subjects were debriefed and told their final scores 


1.4 RESULTS 

Performance scores on test trials were analyzed with a 2 x 2 x 2 between-subjects analysis of 
variance (ANOVA). Overall, performance on test trials was not affected by simulated time 
during training, RT vs. ARTT (F < 1), retention interval, zero vs. two (day (F < /), or the number 
of training trials, 10 or 15 (F < 1). The transition from training to the more demanding real-time 
test trials was assessed using a mixed design ANOVA with trial as the repeated factor. There is a 
significant interaction between training vs. test performance and real-time training vs. ARTT, F 
(12,276) = 6.44, p. < .01 (see Figure 1.1). Mean performance scores during training for students 
using real-time simulation are significantly higher than scores for students trained using ARTT, 
F (1,23) = 43.5, p. < .01. Performance scores of the two groups are not significantly different for 
test trials, F < 1 . 
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Figure 1.1 Mean training and test scores for students trained in real time and ARTT. 

1.5 DISCUSSION 

This experiment was designed to assess the effects of three training variables on performance 
of a demanding, real-time task: simulated time, number of training trials, and retention interval. 
Previous research on ARTT employed a procedure in which training was immediately followed 
by test. It is possible that the effect of ARTT is to adapt the trainee to a rapid pace of responding 
and that this adaptation would fade quickly with time. To evaluate this hypothesis, test trials 
were presented either immediately after training or two days later. Length of the retention 
interval by itself did not affect test performance and did not interact with simulated time. There 
is no evidence from this experiment that skills developed using ARTT are less lasting then real- 
time, training. Also, for this task adding five extra training trials, in real time or using ARTT, did 
not affect performance on test trials. 

Two major effects of ARTT can be seen in Figure 1.1. First, performance scores during 
training for students using ARTT are depressed compared to performance scores of students 
trained using real-time simulation. Performing a challenging task such as the one developed for 
this experiment is made more difficult by performing it quickly. Real-time simulation was more 
effective than ARTT for helping trainees learn to master this task. The second effect of ARTT 
can be seen in the transition from training to test. The ARTT concept was initially developed at 
NASA based on the experience of test pilots that actual flying was more demanding than flying a 
simulator. The idea was that ARTT should reduce the severity of transitioning from the simulator 
to the aircraft by training pilots to fly under higher demand conditions. This hypotheses was 
supported in that performance of students trained using ARTT was not significantly affected 
when they transitioned from training scenarios to the more difficult test scenarios that were 
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presented in real time. The scores of students trained in real time, however, were lower/ 1 or on test 
trials than for training trials. 

Overall, the results of this experiment demonstrate that as an instructional tool, ARTT has 
both costs and benefits. The cost is reduced student performance during training. The benefits are 
improved time efficiency of training and better transition to increasing task demands. Current 
research efforts at Tuskegee University are aimed at developing training strategies that will 
incorporate ARTT into a more comprehensive set of instructional tools. 


1.6 CONCLUSION 

University students were trained to perform an aerial gunnery task with RT or ARTT and 
tested either immediately after training or two days later. Results of this experiment show that 
effects of ARTT did not decrease over a two day retention interval and that ARTT was more 
time-efficient than real-time training in that equal test performance could be achieved with less 
clock-time spent in the simulator. In addition, when students transitioned from training scenarios 
to more demanding test scenarios presented in real time, performance of students trained in real- 
time degraded while the performance of students trained using ARTT was unaffected. 
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2.1 ABSTRACT 


A 3x2 mixed factorial experiment was conducted to compare three different training 
programs for a simulated gunnery task. A participant acted as the weapons officer in the pre- 
programmed simulated flights. The task required identifying friendly and bandit aircraft 
selecting the appropriate weapon, locking and destroying the bandit aircraft, activation of 
emergency counter measures for self-defense. Every participant received 12 minutes or 
familiarization and 36 minutes of training. Every participant was tested immediately after 

training and also seven days after training. _ . _ 

The three training programs are termed as Real Time Training (RTT), Above Real Time 
Training (ARTT), and Real & Above Real Time Training (RARTT). In RTT events on the 
simulator occurred at their normal pace for the complete 36 minutes of training. In ARTT events 
occurred at a pace 1.5 times the normal pace for the complete 36 minutes of training. The 
RARTT consisted of 18 minutes of RTT followed by 18 minutes of ARTT. The test scores 
indicate that the transfer of training is highest in the RARTT condition. In all three programs, t e 
seventh day retention tests revealed no deterioration in the acquired training. 
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2.2 INTRODUCTION 


Above Real Time Training (ARTT) is the training acquired on a real time simulator when it 
is modified to present events faster than normal. The works of Kolf 1 and Hoey suggest that 
above real time training for pilots would lead to significant increase in the effectiveness of flight 
training although it results in reduced simulator fidelity. Kolf 1 notes that “ regardless of type or 
amount of pre-flight simulator training accomplished by the pilot, the actual flight appears to 
take place in a much faster time frame than real time (App. Al).” In an experiment, Kolf 
modified the M2-F3 Lifting Body Simulator to obtain the pace of events 1.5 times their pace in 
real time. Three pilots who had already flown the M2-F3 expressed that the modified simulator 
felt exactly like the aircraft. Hoey 2 reports that measurements of test pilots operating remotely 
piloted vehicles, shows that the stress level, physical level and mental states of the pilots are 
more influenced by the strong sense of responsibility and anxiety, instead of fear or personal 
danger when compared to past data taken in flight. The mental state can be approximately 
simulated without stressful conditions by increasing the simulated rate of time passage. 

While exploring the possible benefits of ARTT, Guckenberger, et. al 3 placed the objectives of 
NASA Dryden Flight Research Center and Air Force Human Systems Center’s Technical 
Planning Integrated Product Team (HSC, TPIT) in perspective. They proposed that the Air Force 
training may derive the following benefits from ARTT: increased task performance, increased 
trainee retention of skills, increased situation awareness, decreased real time work load, 
decreased real time stress, increased rate of skill acquisition, reduced simulator and aircraft 
training time, and more effective emergency procedures training. Crane and Guckenberger 4 have 
reported that pilots trained using ARTT performed emergency procedures and defeated bandit 
aircraft significantly faster than pilots trained in real time. 

Rossi et al 5 trained university students on a gunnery task to compare ARTT verses RTT, 10 
trials verses 15 trials for training, and immediate testing verses testing after two days retention. 
Based on their work, Rossi et al have hypothesized that the use of ARTT as top-off training after 
RTT would result in more effective training. Their hypothesis is addressed in the present study 
by offering mixed RTT and ARTT to university students. In different contexts, mixed training 
had been studied by Guckenberger, Uliano, and Lane 6 and by Crane and Guckenberger 4 . In the 
present study, a 3x2 mixed factorial experiment was conducted to include the use of ARTT as 
top-off training after RT. The three training conditions were RTT, ARTT, and RARTT. In the 
RTT condition, real time training has been provided for the complete training duration. In the 
ARTT condition, above real time training was provided for the complete training duration. In the 
RARTT condition, real time training was provided for the first half of the training duration and 
above real time training was provided for the next half of the training duration, so that ARTT is 
used as top-off training. Three different groups of participants were individually trained and 
tested to compare the effectiveness of the three different training programs. Every participant 
was tested immediately after training and also seven days after training. 

2.3 PARTICIPANTS 

Twenty-one students in a freshman aerospace engineering course at Tuskegee University 
volunteered as participants to obtain the training and to be tested in the research program. The 
participants did not have any formal experience of gunnery tasks or flying. Some participants, 
however, indicated experiences of playing video games. 


II 


2.4 APPARATUS 


The training unit was comprised of two IBM PC compatibles, one used as the instructor s 
station and the other as a student’s station. The student stations was equipped with a joystick and 
a throttle. The flight simulation software for a simplified A- 10 model simulator was installed on 
a Universal Distributed Interactive Simulator (UDIS) platform supplied by ECC International of 
Orlando. The A- 10 like airplane flight was initialized at 10,000 ft above ground level at 250 
knots. In different flights, different scenarios of friendly and bandit airplanes and helicopters 
were encountered while a participant was seated as a weapons officer in front of the compute^ - 
screen. The flight scenarios were developed in house for a prior study conducted by Rossi et al . 
Joy stick and throttle buttons were configured for identifying targets, activating Emergency 
Counter Measures (ECMs), and selecting and firing weapons. After completing each flight the 
statistics on the participants’ performance and percentile score in the flight were displayed on the 
screen. In the RTT every flight was completed in 3 minutes; in ARTT the pace of events was 1.5 
times the normal pace and every flight was completed in 2 minutes. Details on apparatus may be 
seen in Rossi et al 5 . 

2.5 PROCEDURE 

The assignment of participants to each of the three training programs or conditions: RTT, 
ARTT, and RARTT were made randomly. Every participant signed a consent form and 
completed a brief background survey. Every participant was provided with a brief description of 
the apparatus and the task. Every participant completed four familiarization trials each of three 
minutes duration in real time. Every participant in the RTT condition completed twelve training 
flights of three minutes each. Every participant in ARTT condition completed 18 training flights 
of two minutes each. Every participant in RARTT condition completed six flights of three 
minutes each in real time and nine flights of two minutes each in above real time. Nine different 
flight scenarios were used for training; they were randomly presented to a participant, and in 
some cases repeated to complete the training duration of 36 minutes for a participant. Flight 
scenarios were presented either in real time or in above real time mode. In the present study a 
factor of 1 .5 was selected for above real time operation. Therefore, a flight of three minutes in 
real time required two minutes in above real time. Typical familiarization, training, and test 
flights conducted by a participant with respective scores are given in Table 1. 

2.6 RESULTS 

RARTT and one fixed training time of 36 minutes being the between participant factors, and 
two retention periods immediate and seven days being within participant factor. The dependent 
variables have been performance scores, completeness, and retention. All the tests were 
conducted in real time. Only three flights of three minutes duration each were used for testing. 
For immediate testing every participant conducted three flights assigned in random order. For the 
seventh day testing too, every Data were analyzed as a 3x2 mixed factorial experiment, with 
three training conditions RTT, ARTT, and participant conducted three flights assigned in random 
order. On the seventh day, however, before conducting the test flights, every participant 
conducted one familiarization flight of three minutes duration. Twenty-one participants were 
trained in the program. For six of the participants the training of the program was questionable 
either they did not return or they returned later than the seven days, therefore the data for fifteen 
participants have been reported and discussed here. 

Figures 1 and 2 provide the test scores of immediate testing and Figures 3 and 4 provide the 
test scores of the seventh day testing. Figures 1 and 3 have test scores of individual participants 
presented in groups of three to represent three different training conditions. Figures 2 and 4 have 


12 



average scores of five participants in each of the three training programs and average scores of 
three best performances in each of the three training programs. 

It is proposed that an acceptable comparison of training programs or educational programs 
can be based on a few best performing trainees or students. With such an outlook the average 
scores of three best performances in immediate testing and in seventh day testing reported in 
Figures 2 and 4 respectively offer acceptable comparison. The performance clearly suggests that 
the transfer of training is highest in the RARTT program or in the training program with ARTT 
used as top-off training. In all three training programs, the seventh day retention tests reveal no 
deterioration in the acquired training. Analysis of variance 7 is applied to five participants in each 
program and to the three best performances in each program. The variance values and the F 
ratios are presented in Table 2 a & b. When the number of participants are the same in two 
different training programs, then the F ratio is the ratio of variance of test scores of all the 
participants in the two programs to the average of variance of test scores of participants 
considered program-wise. For five participants in each program, the F ratios are less than one; 
they indicate that the three different training programs do not offer significantly different transfer 
of training. The F ratios based on three best performances, however, reveal some significant 
differences. For comparison between RTT and ARTT, the F ratio of less than one suggests that 
RTT alone and ARTT alone do not offer significantly different transfer of training. For 
comparison between RARTT and RTT, the F ratio is 1.83 and for comparison between RARTT 
and ARTT, the F ratio is 1.87. Therefore, from analysis of variance applied to the same day tests, 
we may infer that the use of ARTT as top off training after RTT offers significantly better 
training in comparison to ARTT alone or RTT alone. The F ratios of the seventh day tests are all 
less than one. The large variance in the seventh day tests of RTT is noticeable. It would, however 
be worthwhile to verify the inference based on same day tests by conducting an experiment 
requiring relatively more complex training tasks and by acquiring sample data with a larger 
number of participants. 

In addition to the inference based on the best three performances, the traditional statistical 
inferences based on the sample size of five are obtained by using the sign test. Relevant 
quantities for the sign test are provided in Table 3. For the sign test, the null hypothesis implies 
no significant difference between the two modes of training which are compared. For the three 
different comparisons shown in Table 3, the null hypothesis may be rejected if the error a is 
allowed to be 0.18. The sign test inference and analysis of variance inference are same for 
comparing RTT and RARTT and for comparing ARTT and RARTT, but they are not same for 
comparing RTT and ARTT. 
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TABLE 2.1 


Typical Familiarization and Test Flights With Respective Scores 



Fam. Flights 

Training Flight 

Test Flight 

Enemies encountered 

9 

11 

26 

Enemies shot down with correct 
weapon* 

8(16/18) 

8(16/22) 

16(32/52) 

Enemies shot down with wrong 
weapon* 

1 (-2/0) 

0 

0 

Enemies missed* 

0(0/0) 

3(-6/0) 

10(-20/0) 

Friendlies encountered 

6 

10 

12 

Friendlies shot down* 

1 (-4/0) 

0(0/0) 

0(0/0) 

ECM required 

3 

6 

9(9/9) 

ECM activated* 

3(3/3) 

5(5/6) 

9 

IFF inquiries on friendly 

0 

0 

1 

IFF inquiries on enemy 

0 

0 

0 

Final objective met* 

1(5/5) 

1(5/5) 

1(5/5) 





Total scored points 

18 

20 

26 

Maximum possible score 

26 

33 

66 

Percent score 

69.5 

60.6 

39.39 





* Earned Score/Maximum possible score are given in parentheses i 
































































FIG. 2.4 SEVENTH DAY AVERAGES AND TOP 3 AVERAGES 
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Table 2.2a: Analysis of Variance For 3 Groups Based on Top 3 Performances 

Same Day Test 
RTT, ARTT, RARTT 


Training 

Type 

Five Scores For Each Type 

Average 

Variance 
Per type 

Average 

OfVar. 

Total Var. 
Per group 
Of two 
types 

F Ratios/ 
Retention 

RTT 

29.6 

26.3 i 

18 

7.08 

10.6 

18.31 

94.36 

110.19 

101.63 

0.92 

ARTT ! 

30.2 

20.1 

13.3 

0.57 

9.09 

14.66 

126.01 



same day 







RTT 

29.6 

26.3 

18 

7.08 

10.6 

18.31 

94.36 

185.55 

172.63 

0.93 

RARTT 

36.4 

31.8 

31.6 

23 

-4.9 

23.57 

276.74 



same day 







ARTT 

30.2 

20.1 

13.3 

0.57 

9.09 

14.66 

126.01 

201.38 

201.03 

i ^ 

RARTT 

36.4 

31.8 

31.6 

23 

-4.9 

23.57 

276.74 



same day 




7th Day Test 


BBBjW 

RTT 

45.32 

34.64 1 

15.15 


13.69 

24.39 

217.22 

167.26 

148.71 


ARTT 

34.44 

33.99 

28.52 


10.34 

24.76 

117.3 



QX5SHH 






RTT 

45.32 

34.64 



13.69 

24.39 

217.22 

152.385 

141.62 

0.93 ~1 

RARTT 

37.89 

37.18 

30.44 


15.51 

29.1 

87.55 



7 th day 







ARTT 

34.44 

33.99 

28.52 

16.5 

10.34 

24.76 

117.3 

102.425 

96.29 

0.94 

RARTT 

37.89 

37.18 

30.44 

24.5 

15.51 

29.1 

87.55 



7th day 


Table 2.2b: Analysis of Variance For 3 Groups on Top 3 Performances 

Same Day Test 
RTT, ARTT, RARTT 


Training 

Type 

Top 3 Scores For Each Type 

Average 

Variance 
Per type 

Average of 
Variance 

Total Var. 
Per group 
Of two type 

F Ratios/ 
Retent. 

RTT 

29.58 

26.32 

18 

24.63 

35.66 

54.19 

46.85 


ARTT 

30.24 

20.13 

13.29 

21.22 

72.72 



BBMHII 










RTT 

29.58 

26.32 

18 

24.63 

35.66 

21.53 

39.5 


RARTT 

36.39 

31.77 

31.59 

33.25 

7.4 











WBSSSm 


ARTT 


20.13 

13.29 

21.22 

72.72 



1.87 



31.77 

31.59 

33.25 

7.4 



same day 





RTT 

45.32 

34.64 

15.15 

31.77 

234.03 

122.45 

98.07 


ARTT 

34.44 

33.99 

28.52 

32.32 

10.86 



7th day 




RTT 

45.32 

ihism 

15.15 

31.77 

234.03 

125.47 

103.98 

0.82 

RARTT 

37.89 

mmm 

30.44 

35.17 

16.91 







ARTT 

34.44 

33.99 

28.52 

32.32 

10.86 

13.89 

13.55 

0.98 

RARTT 

37.89 

37.18 

30.44 

35.17 

16.91 


----- 

7th day 


19 







































































Table 2.3: Sign Test 


TRAINING 

i 

2 

3 

4 

5 

x in ••••x* 

••value to 
reject 
null 

RTT (yl) 

7.08 

26.32 

18 

29.58 

10.56 



ARTT (y2) 

30.24 

13.29 

0.57 

20.13 

9.09 



Sign of 










I 

+ 

+ 

+ 

+ 

1.34* 

0.18 

RARTT (yl) 

31.77 

-4.9 1 

36.39 

31.59 

22.98 



RTT (y2) 

7.08 

26.32 

18 

29.58 

10.56 ^ 



Sign of 








yi-yg 

+ 

- 

+ 

+ 

+ 

1.34- 

0.18 

RARTT (yl) 

31.77 

-4.9 

36.39 

31.59 

22.98 



ARTT (y2) 

30.24 

13.29 

0.57 

20.13 

9.09 



Sign of 








yt-yj 

+ 

+ 

+ 

+ 

1.34- 

0.18 


2.7 CONCLUSION 

A study on the comparison of three different training programs for a simulated gunnery task 
is reported. The three training programs are termed as RTT, ARTT and RARTT In RTT or real 
time training, events on the simulator occurred at their normal pace for the complete 36 minutes 
of training In ARTT or above real time training events occurred at a pace 1.5 times the normal 
pace for the complete 36 minutes of training. The RARTT consisted of 18 minutes of RTT and 
18 minutes of top-off training in the ARTT mode. The data of five participants m each mode of 
training were reported. The Analysis of variance, however, has been applied to the three best 
performances in each training mode. It is inferred that the use of ARTT as top-off training after 
RTT offers significantly better training in comparison to ARTT alone or RTT alone. The results 
are encouraging toward conducting a similar experiment on a relatively more complex task with 
a sample of a larger number of participants. In all three training programs, the seventh day 
retention tests revealed no deterioration in the acquired training, provided a brief re- 
familiarization with the task is allowed. 
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